GENERALIZED DUMONT-FOATA POLYNOMIALS AND 
ALTERNATIVE TABLEAUX 
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^ ' Abstract. Dumont and Foata introduced in 1976 a three-variable symmetric 

_ O , refinement of Genocchi numbers, which satisfies a simple recurrence relation. A 

| /^ i ' six-variable generalization with many similar properties was later considered by 

Dumont. They generalize a lot of known integer sequences, and their ordinary 
^\1 ' generating function can be expanded as a Jacobi continued fraction. 

We give here a new combinatorial interpretation of the six- variable polynomials 

in terms of the alternative tableaux introduced by Viennot. A powerful tool to 

enumerate alternative tableaux is the so-called "matrix Ansatz" , and using this 

Shi/ ' we show that our combinatorial interpretation naturally leads to a new proof of 

(~| , the continued fraction expansion. 

-)— > 



o 



271" 



2r) 

X ■ tan 



CN ■ 1. Introduction 

>: 

t^^ , The unsigned Genocchi numbers {G2n}n>i can be defined through their gener- 

^ I ating function: 

^/ 

O ■ "^ (2n) V2 

O: n=l ^ ^ 

They are related with even BernouUi numbers i?2n by G2n = 2(4" — l)|i?2n|, and 
they have a wide range of combinatorial properties [4, 10, 12, 14]. In the context 
^ ■ of previous work by Carlitz, Riordan and Stein, an extension of these integers was 

H ! proposed by Dumont and Foata [5]. It is defined by the recurrence Fi{x, y, z) = 1 

and 

(2) Fn{x, y, z) = {x + y){x + z)Fn-i{x + 1, y, z) - x^Fn-i{x, y, z). 

They show that the polynomial Fn is symmetric in x, y, and z, with non-negative 
coefficients, and such that F„(l, 1, 1) = G2n+2- Another nice property is that the 
generating function Xl^i ^nt^ can be expanded as a J-fraction (more precision 
will be given further in this introduction). Gessel and Zeng [7] showed that F„+i is 
the nth moment of some orthogonal polynomials known as continuous dual Hahn 
polynomials, which are an important sequence in the Askey- Wilson hierarchy [8]. 



Partially supported by the grant ANR08-JCJC-0011. 

1 



2 MATTHIEU JOSUAT- VERGES 

A further generalization of Genocchi numbers with many similar properties was 
defined by Dumont in terms of some combinatorial objects called escaliers [4]. It is 
a sequence of six- variable polynomials r„(x, y, z, x, y, z), or just r„ for short. They 
can be characterized by a recurrence relation which generalizes (2), and has been 
obtained independently by Randrianarivony [10] and Zeng [14]. For brevity, let r+ 
denote r„(x + l,y, z,x + l,y,z). 

Definition 1.1. The generalized Dumont-Foata polynomials are defined by Fi = 1 
and 

(3) F„ = (x + z){y + x)F+_i + (x{y - y) + x{z - z) - xx)F„_i. 

Quite a lot of known integer sequences appear as specializations of F„ [10, 11, 
14]: Genocchi numbers, median Genocchi numbers, Euler numbers, median Euler 
numbers. Springer numbers. This polynomial F„ generalizes F„ since we have 
F„(a;, y, z, x, y, z) = Fn{x, y, z). 

Dumont [4] conjectured that we have the following J-fraction for ^ F^t": 

oo 

(4) J2^nr = 



"=l 1 - 6nt — 



bit 



Aot 
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where the parameters 6„ and A„ are defined by: 

bn = {x + n){y + n) + {y + n){z + n) + {z + n){x + n) — n{n + 1), 

if] 

Xn = n{x + y + n — l){y + z + n — l){z + X + n — 1) . 

This was proved independently by Randrianarivony [10] and Zeng [14] (and of 
course this implies the J-fraction expansion for ^F„t"). More precisely, Randri- 
anarivony's method consists in the study of a Stieltjes tableau and Zeng's method 
consists in calculations of Hankel determinants. 

The main goal of this article is to give a new combinatorial interpretation of F„ 
in terms of alternative tableaux [9, 13] and six statistics on them, and obtain as 
a consequence a new proof of the continued fraction expansion (4). Alternative 
tableaux were introduced by Viennot [13] in the context of a model of statistical 
physics called Partially Asymmetric Simple Exclusion Process (PASEP), and previ- 
ous work of Corteel and Williams [2]. The "matrix Ansatz" first appeared in [3], as 
a way to obtain the stationary distribution of the PASEP. In the combinatorial con- 
text, it is a method to enumerate these alternative tableaux in terms of operators 
satisfying certain relations. We will also describe an analog of the matrix Ansatz to 
enumerate escaliers, which are the combinatorial objects used by Dumont to define 
r„ in [4]. 
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This article is organized as follows. In Section 2 we give definitions and known 
facts about alternative tableaux and the matrix Ansatz. In Section 3 we prove the 
new combinatorial interpretation of r„ in terms of alternative tableaux using the 
recurrence (3). Section 4 contains our new proof of the continued fraction expan- 
sion (4). In Section 5, we describe the analog of the matrix Ansatz to enumerate 
escaliers. 

2. Alternative tableaux 

Throughout this article we use the French convention for Young diagrams, and 
Young diagrams may contain rows or columns of size 0. Any Young diagram is 
characterized by its upper-right boundary, which is a sequence of unit steps going 
left or going down. We will encode this sequence by a word in the two letters D 
and E, so that D corresponds to the step — )■ and E corresponds to the step J,. For 
example, DDEDE is the Young diagram with two rows of respective lengths 2 and 
3. 

Definition 2.1. Let A be a Young diagram. An alternative tableau of shape A 
is a filling of A such that each cell is either empty, contains an arrow ^ or an 
arrow J,, and each arrow has a clear view to the boundary. More precisely, all cells 
below a J, in the same column (or to the left of a <(— in the same row) are empty. 
A column (respectively, row) of an alternative tableau is free if it contains no J, 
(respectively, no ^). We denote by fr(T) (respectively, fc(T)) the number of free 
rows (respectively, free columns) of T. See Figure 1 for examples. We use here the 
notation with arrows as introduced by Nadeau [9]. 
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Figure 1 . Examples of alternative tableaux. 

Alternative tableaux of a given shape can be enumerated via a method called 
matrix Ansatz. This method appeared in the context of a model of statistical 
physics (the partially asymmetric simple exclusion process), where it is used to 
derive the stationary probabilities of any state of the process. 



Proposition 2.2 (Corteel- Williams [2]). Let {W\ be a row vector, \V) a column 
vector, and D and E matrices such that: 

{Q){W\V) = 1, {W\E = x{W\, D\V) = y\V), and DE-ED = D + E. 
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Let w be a word in the two letters D and E, then we have 

(7) {w\w\v) = Y,^'-'^'^^y'"^'^^ 

T 

where the sum is over alternative tableaux T of shape w. 

The result of Corteel and Williams was actually stated in terms of permutation 
tableaux, which are slightly different objects. We refer to Viennot [13] and Nadeau 
[9] for the corresponding statement in terms of alternative tableaux, and for the 
bijection between permutation tableaux and alternative tableaux. We have chosen 
to use alternative tableaux in this work because of their symmetry. Indeed, there is 
an elementary involution on alternative tableaux which is conjugation. To conjugate 
a tableau, take the image of the whole picture with respect to the South- West to 
North-East axis symmetry (in particular, the <— and I are exchanged). See [9] for 
details. 

Note that relations (6) ensure that (VF|t(;|K) is well-defined and can be computed 
explicitly. Indeed, we can use DE — ED = D + E to obtain some coefficients Qj 
such that w = Y2i , CijE'^D\ and from the other relations we can obtain (VF|t(;|y). 
We refer to [2, 1] for more details. 

Although not necessary to compute (VF|zi;|y) for a given word ly, it is useful to 
have explicit matrices satisfying the PASEP matrix Ansatz. It can be checked [3] 
that the following N x N-matrices: 
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2/ + 1 
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y + 2 
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2/ + 3 
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V(o) 

satisfy DE — ED = D + E. They are essentially a particular case of matrices 
defined by Derrida & al [3] in the context of the PASEP. As for the vectors, we 
can take {W\ = (1,0,0,...) and \V) = (1,0,0,...)*, and all relations in (6) are 
satisfied. 



3. The new combinatorial interpretation of r„ 

It is known [13] that G'2n+2 is the number of alternative tableaux whose shape is 
the staircase with n rows and columns, i. e. the Young diagram corresponding to the 
word (DE)^. In [1], we have given three statistic in staircase alternative tableaux 
to give a combinatorial interpretation oi Fn{x,y, z). These are: the number of free 
rows, the number of free columns, and the number of corners containing -(— or J,. 
Here, we give six statistics for the more general case of r„. Another difference 
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is that in [1], the combinatorial interpretation was derived from the J-fraction for 
^ F„t", but here we use the recurrence relation (3) to prove the result. 

Definition 3.1. A column (respectively, row) of an alternative tableau is empty if 
it contains no J, nor -(— . Let T be an alternative tableau. We denote by: 

• emr(T), the number of empty rows in T, 

• fnc(T), the number of free non-empty columns in T, 

• dco(r), the number of corners containing a J, in T, 

• fnr(T), the number of free non-empty rows in T, 

• emc(T), the number of empty columns in T, 

• lco(T), the number of corners containing a ■(— in T. 

Moreover let Tn be the set of alternative tableaux whose shape is the staircase 
Young diagram with n rows and n columns. 

For example, the rightmost tableau in Figure 1 is in Ts, and the six statistics 
that we have just defined are respectively 2, 2, 2, 0, 1, 0. The main new result of 
this article is the following: 

Theorem 3.2. For any n> 1, we have 



(9) 



ny^i y-i z, X, t/, Zj 



TeTn-i 



^emr{T) ^fnc(T) ^dco(T) ^fnr(T) ^emc(T) -lco(T) 



Proof. Both sides are equal to 1 when n = 1, so it suffices to show that the right- 
hand side satisfies the recurrence relation (3). We distinguish six kinds of tableaux 
in the set Tn-i, according to the content of their leftmost column and upper left 
corner. Assuming that the theorem is true for n — l, we will show that these six 
kinds of tableaux have generating functions which add up to the right-hand side of 
(3). This is summarized in the following table. 
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For example we will show that the tableaux of the fourth kind (case 4), i.e. those 
having an empty leftmost column, have generating function xyVn-i- The three cells 
containing a x in this table do not correspond to any tableaux. 
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Figure 2. Recursive construction of staircase alternative tableaux. 



• Case 1. When the upper- left corner contains a J,, there is no other arrow in the 
leftmost column. This corresponds to the first picture in Figure 2. In this first 
kind of tableaux, the topmost row is free non-empty, and the upper left corner 
contains a J,, so this gives a factor xz. After removing the leftmost column, 
there can remain any tableau in 7^_2, hence the factor r„_i. So the first kind 
of tableaux gives indeed the term xzTn-i- 

• Case 2. There is a factor yz since we assume that there is a ^ in the upper left 
corner, and that the leftmost column is free and non-empty. These tableaux can 
be obtained the following way: consider any tableau T in Tn-2, add a column 
to its left with a -(— in the topmost cell of the added column. We color some of 
the other cells in the added column in gray as in the second picture in Figure 2, 
such that a cell is colored if there is no ■(— to its right. Then, decide whether 
each gray cell is empty or contains a ^— . All tableaux of the second kind can be 
obtained this way, and the gray cells are in correspondence with the free rows of 
T. At the level of generating functions, this amounts to substitute x with x + 1 
and X with x + 1. Indeed, an empty (respectively, free non-empty) row remains 
so if we add nothing in the gray cell, but becomes non-free if we add a ^. 

• Case 3. This corresponds to the second picture in Figure 2, but with the 
assumption that there is a J, in one of the gray cells. Let us consider the set S 
of tableaux of the second kind such that there is at least a <— in some gray cell. 
This set has generating function yz{T^__^ — Tn-i), indeed we have already yzT^_i 
for the all tableaux of the second kind and the term —yzTn-i removes the cases 
where all gray cells are empty. Then, there is a bijection between this set S and 
the tableaux of the third kind. Indeed, let T E S, consider the bottommost <(— 
in the leftmost column of T, and replace this -(— with a J,. This way, we obtain 
exactly the tableaux of the third kind. Replacing the ^ with J, gives a factor 
xy~^ at the level of generating functions. Thus we obtain xz(T^_-^ — r,„_i) for 
the third kind of tableaux. 

• Case 4. This is similar to case 1, and corresponds to the third picture in 
Figure 2. Here, removing the first column gives a factor xy since we assume 
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that the leftmost column is empty, and hence the upper row is empty. There 
can remain any tableau in 7^_2, hence the factor r„_i. Thus we obtain xyTn-i 
for the fourth kind of tableaux. 

• Case 5. This corresponds to the fourth picture in Figure 2, with the assumption 
that the gray cells contain no J, and at least a •(— . Here the gray cells are 
obtained exactly as in case 2 above. Proceeding similar to case 3 above, we 
obtain xy{r^_i — r„_i) for the fifth kind of tableaux. 

• Case 6. There is a bijection between the fifth kind and the sixth kind of 
tableaux, similar to the bijection used in case 3. From a tableau of the fifth 
kind, consider the bottommost -(— in the leftmost column, and replace it with 
a I. Replacing the ^ with ], gives a factor xy~^ at the level of generating 
functions. Thus we obtain xx(r^_^ — r„_i) for the sixth kind of tableaux. 

Adding the six terms in the above table, we get the right-hand side of (3). This 
shows that the right-hand side of (9) satisfies the same recurrence as r„, and 
completes the proof. D 

As previously mentioned, there is a simple bijection between alternative tableaux 
and permutation tableaux and it is possible to derive a combinatorial interpretation 
of r„ in terms of permutation tableaux, but the result is much more natural with 
the alternative tableaux. In particular, the conjugation of alternative tableaux gives 
an easy way to prove a symmetry property of F^, which has been first given by 
Randrianarivony [10] and Zeng [14]. 

Proposition 3.3 ([10, 14]). For any permutation a = u,v,w ofx,y,z we have: 

• if a has signature 1 , thenTn{u,v,w,u,v,id) = Tn{x,y, z,x,y,z), 

• if a has signature -1, thenTn{u,v,w,u,v,w) = Tn{x,y,z,x,y,z). 

In particular, F„(x, y, z) = F„(a;, y, z, x, y, z) is symmetric in x, y, and z. 
Proof. From the recurrence relation (3) , we have 

(10) F(a;, y, z, x, y, z) = F(x, z, y, x, z, y). 

From the combinatorial interpretation in (9) and using the conjugation of alterna- 
tive tableaux, we have 

(11) r(x, y, z, X, y, z) = T{y, x, z, y, x, z). 

All symmetries given in the can be obtained by combining (10) and (11). □ 

It is rather curious that one symmetry is obvious on the combinatorial interpreta- 
tion, and another one in the recurrence relation. In the model in terms of escaliers 
(see Section 5), only one symmetry is obvious, and it is both in the combinatorial 
interpretation and in the recurrence relation. Note that any symmetry of a gener- 
ating function necessarily appear in the coefficients of its expansion as a J-fraction, 
and indeed it is straightforward to check that the coefficients 6„ and A„ defined in 
(5) have the same symmetries as F^. 
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4. The continued fraction expansion 

In this section, we show that r„ can be calculated via the matrix Ansatz, and 
we derive as a consequence a new proof of the continued fraction expansion for 
^r„t". We consider the matrix 

(12) M = ED + {z + x- x)D + {z + y- y)E + (y - y){x - x)I, 

where D, E are defined in (8), and / is the identity matrix. It turns out that we 
can exploit Proposition 2.2 to obtain the following: 

Proposition 4.1. For any n > 0, we have 

(13) T^+,{x,y,z,x,y,z) = {W\M''\V). 

To prove this, we need a few helpful definitions and lemmas. 

Definition 4.2. Let T* be the set of pairs (T, X) where T ^ Tn and X is a subset 
of the empty rows and columns of T. Such a pair (T, X) is called an extended 
tableau, and will be represented the following way: from a picture of T, each row 
or column in X is distinguished by a dashed line going through it. See Figure 3 for 
some examples. Given U = (T,X) G 7^*, we define 

• hr(t/) as the number of dashed rows, 

• hc(f/) as the number of dashed columns. 

For any statistic "stat" on alternative tableaux, and U = (T, X) G 1~* we define 
stat(t/) = stat(T). For any extended tableau U, we define the weight w{U) as 

(14) 



^(^U^ ^ ^emT[U)-hT(U}ytnc[U)^dco{U)^tnT(U}yerac{U)-hc(U)^lco(U}^^ _ ^)Mt^)(^^ _ y^ 



hc((7) 



^fc(l/)-hc(C/)_^dco(l/)-fr(C/)-hr(l/)-lco(l/)(^^ _ ^)hr(C7)(^^ _ ^)hc(C/)^ 



the latest equality following from emr(r) + fnr(r) = fr(T) and emc(r) + fnc(T) 
fc(r) for any alternative tableau T. 
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Figure 3. Example of two extended alternative tableaux. 



Lemma 4.3. We have: 



U€T* 



T&Tn 
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Proof. In the sum J2ueT* '"^(^); ^^ have distinguished two kinds of empty rows 
(dashed or non-dashed) with respective weights x — x and x instead of one kind 
of empty row with weight x. Similarly we have distinguished two kinds of empty 
columns (dashed or non-dashed) with respective weights y — y and y instead of one 
kind of empty column with weight y. By an elementary argument, it is clear that 
these distinctions do not change the generating function. D 

Definition 4.4. Let (T, X) be an extended tableau in T*. The profile of (T, X) is 
the sequence («!,...,«„), where: 

• Zfc = 1 if the kth corner of T is empty, 

• ifc = 2 if the kth corner of T contains a -(— , 

• ifc = 3 if the fcth corner of T is in a dashed row but not in a dashed column, 

• ifc = 4 if the fcth corner of T contains a J,, 

• ifc = 5 if the kth corner of T is in a dashed column but not in a dashed row, 

• ifc = 6 if the kth corner of T is in a dashed column and in a dashed row. 

Here the corners are numbered from the upper left one to the lower right one. For 
example, the first extended tableau in Figure 3 has profile (1,5,1,4,6), and the 
second one has profile (5, 2, 3, 1, 4). 

Lemma 4.5. Let Mi, . . . , Mq be the matrices 

Mi = ED, M2 = zD, M3 = {x-x)D, 

(16) 

M^ = zE, M5 = {y-y)E, M, = {y - y){x - x)I. 

For any (zi, . . . , i„) G {1, . . . , 6}", we have 

(17) Y.'^{U) = {W\M,,...M,JV), 

u 

where the sum is over extended tableaux U of profile (zi, . . . ,i„). 

Proof. Let w be the word obtained from ii . . .in through the substitution 1 i— )■ ED, 
2 I— 7- D, 3 I— 7- D, 4 I— 7- i?, 5 I— 7- i?, 6 I— 7- e (e being the empty word). The main 
point is that there is a bijection (p between elements in T* of profile {ii, . . . ,in) 
and alternative tableaux of shape w. Indeed, to build an extended tableau, once 
the contents of the corners are specified it remains only to choose an alternative 
tableau of a smaller shape. More precisely, the bijection can be done the following 
way: 

• for each empty corner of the extended tableau, remove the corresponding cell 
in the Young diagram, 

• shrink each dashed row or column, 

• for each corner of the extended tableau containing a <— (respectively, J,), shrink 
the row (respectively, column) containing it. 
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Figure 4. Images of extended tableaux by the map 



See Figure 4 for an example, where we give the image of the two extended tableaux 
in Figure 3. 

The weight of an extended tableau U of profile (ii, . . . , z„) is the product 



y''z''x'z'^{x-xy{y~yy, 



where: 



• a is the number of non-dashed free columns in U, 

• h = dco{U) is the number of 4's in (zi, . . . , z„), 

• c is the number of non-dashed free rows in [/, 

• d = \co{U) is the number of 2's in (zi, . . . , z„), 

• e = hr([/) is the number of 3's plus the number of 6's in (ii, . . . , i„), 

• / = hc([/) is the number of 5's plus the number of 6's in (zi, . . . , z„). 

An important property of the bijection (f> is that the free rows (respectively, columns) 
of (t){U) are in correspondence with non-dashed free rows (respectively, columns) of 
U. It follows that 



'li 



J2w{U) 



z^zHx - x'"^- 



{y-yy^x-'-^'y 



-fr(T)„fc(r) 



u 



where the first sum is over extended tableau of profile {ii, . . . ,in) and the second 
one is over alternative tableau of shape w. 

Now, examine the product Mj^ . . .Mj^. The factors D and E in this product 
readily gives the word w, and the other factors readily gives z^z'^{x — xY{y — y)-^ , 
so: 



(19) {W\Mi, . . . MijV) = zh\x - xfiy - yy{W\w\V). 

Using Proposition 2.2, the result follows from (18) and (19). 
Now, we can prove Proposition 4.1. 



D 



Proof. Since M = ^j=iMj, the expansion of M" is also the sum of all products 
Mj^ . . . Mj„ where (ii, . . . , in) runs through the set {1, . . . , 6}". Hence, 



(20) 



{W\M^\V)= Y. {W\M,,...M,JV). 

(il,-,«n)e{l,...,6}" 
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Using Equation (17) in Lemma 4.5, this gives 

(21) (iy|M"|y) = J2 ^(^)- 

Using Equation (15) in Lemma 4.3 and Theorem 3.2, this is equal to r„+i. D 

From the definitions of D and E in (8), the matrix M defined in (12) can be 
calculated explicitly and we obtain the following statement. 

Proposition 4.6. The matrix M = (M(j)jjgN is tridiagonal, and such that for 
any i > we have 

(22) Mi^i = bi and M,,,+iM,+i,i = \+i, 
where hi and Aj are defined in (5) . 

Proof. We have 

Mi^i = Ei^iDi^i + ^i,i_iA-i,j + {z + x- x)Di^i + {z + y- y)Ei^i + {y - y){x - x) 

= {x + i){y + i) + {y + X + i — l)i + {z + X — x){y + i) 

+ {z + y - y){x + i) + {y - y){x - x) 

= xy + yz + zx + i[x + y + z + X + y + z) + i(2i — I) = bi. 

We have also 

Mi^i+i = Ei^iDi^i+i + (z + x- x)Di^i+i 

= {x + i){i + I) + {z + X - x){i + 1) = {x + z + i){i + I), 

and 

Mi+i^i = Ei+i^iDi^i + {z + y- y)Ei+i^i = {y + x + i){y + i) + {z + y - y){y + x + i) 

= {x + y + i){z + y + i). 

Hence, M, j+iMj+i j = Aj+i. It is straightforward to check that other coefficients in 
M are 0, and this completes the proof. D 

As a direct consequence of Propositions 4.1 and 4.6, let us give a new proof of 
the continued fraction expansion given in (4). First, note that (H^IM"!!/) is the 
top-left coefficient (M")o,o of the matrix M". This coefficient can be obtained by 
expanding the product M" and we obtain 

(23) {WW\V) = Yl ^o,nM,,, . . . M,„_,,„_,M,„_,,o. 

Ji,...,i„_l>0 

Since the matrix M is tridiagonal, we can restrict the sum to the case where two 
successive indices differ by at most 1, i.e. \ij — ij+i\ < 1 for any j e {0, . . . , n — 1} 
where io = in = 0. These indices thus define the successive heights of a Motzkin 
path. Then (23) shows that r„_|_i can be seen as the generating function of Motzkin 
paths of n steps, with a weight hi for a level step at height i, and a weight Aj for 
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a step /^ between height z — 1 and %. By a standard argument [6] this imphes the 
continued fraction given in (4). 

5. The matrix Ansatz for escaliers 

In the previous section, we have apphed the hnk between alternative tableaux and 
matrices D and E satisfying DE — ED = D + E to obtain the continued fraction. 
In this section, we consider escaliers, which are the combinatorial objects used by 
Dumont [4] to define r„. We will show that these objects can be enumerated by a 
similar method, but with matrices B and A satisfying BA — AB = A + I. 

We will denote a Young diagram by a word in B and A in the same way that 
we did with D and E {B and A respectively correspond to steps — )■ and J, in the 
North-East boundary of the Young diagram). 

Definition 5.1. A surjective pretableau is a partial filling a Young diagram with 
X , such that there is at least one x in each row and at most one x in each column. 
A surjective tableau is a surjective pretableau such that there is exactly one x in 
each column. An escalier (of size n) is a surjective tableau of shape {BBA)"'. See 
Figure 5 for some examples. 
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Figure 5. Examples of surjective pretableaux of shape (BBA)^. 

The definition of r„ by Dumont [4] is given in terms of escaliers of size n and 
six statistics on them. There is an obvious bijection between escaliers of size n and 
surjective pretableau of shape {BBA)"'~^ (remove the bottom row of the escalier), 
so that Dumont 's definition is equivalent to the following (as mentioned in the 
introduction, this is also known to be equivalent with Definition 1.1). 

Definition 5.2. A co-corner of the Young diagram {BBA)"' is a cell which is the 
left neighbor of a corner (for example the upper- left cell is a co-corner). Let Sn be 
the set of surjective pretableau of shape (BBA)^. A column is empty if it contains 
no X . A X is doubled if there is another x in the same row. We denote by: 

• mi(T), the number of empty columns of odd index, 

• fd(T), the number of corners containing a doubled x, 

• snd(T), the number of co-corners containing a non-doubled x, 

• mp(T), the number of empty columns of even index, 

• fnd(T), the number of corners containing a non-doubled x, 

• sd(r), the number of co-corners containing a doubled x. 
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Eventually, r„ can be defined as 

^,mi(T)„fd(T) ^snd{T) -mp(T) -fnd(T) -sd(T) 
TeS„ 



(24) r„ = ^ ^....,.,^..,.,^....,.,^...^,.,^....,.,^. 



For example, the values of the six statistics on the first surjective pretableau in 
Figure 5 are 1, 1, 1, 2, 1, and 0. As for the second one, the values are 0, 1, 2, 
2, 0, and 1. The fact that these objects also follow the recurrence (3) is seen by 
distinguishing several kinds of elements in Sn-i according to the content of the 
bottom row [10, 14]. 

The analog of the matrix Ansatz for escaliers is given in the following proposition. 
The proof is similar to the case of alternative tableaux, and various examples of 
this kind of results were given in [1]. 

Proposition 5.3. Let {W\ be a row vector, \V) a column vector, and A and B 
matrices such that: 

(25) {W\V) = l, {W\A = Q, B\V) = Q, BA-AB = A + I. 

Let w he a word in the two letters B and A, then the number of surjective tableaux 
of shape w is {W\w\V). 

Proof. This is done by a recurrence on the number of cells in the Young diagram. 
If there is no cell, then vu = A^B^ for some i and j, so (VF|i(;|V^) equals if z > 
or j > and 1 otherwise. Since there is at least a x in each row and column of 
a surjective tableau, there is no such tableau of shape A'^B^ if z > or j > 0, but 
we do have the "empty" surjective tableau in the "empty Young diagram" when 
i = j = 0. 

Next, consider a word w which is not in the form A^BK It means we can f actor ize 
iimw = WiBAvu2, and the factor BA corresponds to a corner of the Young diagram. 
We can distinguish three kinds of surjective tableaux of shape w, depending on the 
content of this corner. 

• If the corner is empty, we can remove it and obtain any surjective tableaux of 
shape wiABvu2- By the recurrence assumption, their number is {W\wiABw2\V) . 

• If the corner contains a doubled x , we can delete the corner and its column, and 
obtain any surjective tableau of shape wiAw2- Their number is {W\wiAw2\V). 

• If the corner contains a non-doubled x , we can remove the corner and its row 
and column, and obtain any surjective tableau of shape WiVU2- Their number is 

{W\WiW2\V). 

It follows that the number of surjective tableaux of shape w is 

{W\wiABw2\V) + {W\wiAw2\V) + {W\wiW2\V) = {W\wiBAw2\V) = {W\w\V), 
and this completes the recurrence. D 
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It is easily checked that the following N x N-matrices: 



/O 1 



(26) 
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1 2 
2 3 



V(0) 



(0)\ 
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/o 



V(o) 





1 
1 



(0)\ 



7 



satisfy BA — AB = A + I. We keep the definition of {W\ and \V) as in the previous 
sections, since this also ensures that we have (Vr|yl = and B\V) = 0. 

To see how to use this result in the case of r„, let us begin with the particular 
case y = z = y = z = l. We know that the number of surjective tableaux of shape 
(BBA)" is {W\{BBA)"'\V). If we want to count surjective pretableaux, we have to 
authorize empty columns, and this is done by replacing B with B + I. Indeed, in the 
expansion of the product {W\{{B + I){B + I)A)'^\V), the choice oi B or I in some 
factor corresponds to the choice of leaving a column empty or not. So the number 
of surjective pretableaux of shape (BBA)" is {W\{{B + I){B + I)A)''\V). If we want 
to follow the empty columns of odd (respectively, even) index by the parameter x 
(respectively x), it suffices to mark the terms /, and we obtain r„+i(x, 1, 1, x, 1, 1) = 
{W\{{B + xI){B + xJ)A)"|y). 

As for the general case, in the same way that we have obtained Proposition 4.1 
from the combinatorial interpretation in terms of alternative tableaux, we can ob- 
tain the following from the combinatorial interpretation in terms of escaliers. 



.ere N is the 



Proposition 5.4. For any n > 0, we have Tn+i = {W\N"\V) wh 
matrix 

(27) A^ = A{B + xI){B + xI) + yz{A + I) + {zI + zA){B + xI) + {yI + yA){B + xI). 

As in the previous section, we need some helpful definitions and lemmas. 

Definition 5.5. The profile of T G iS„ is the sequence (zi, . . . , ■?„), where 

• Zfc = 1 if the kth co-corner and A;th corner are empty, 

• ^fc = 2 if the kth co-corner and kth corner both contain a x , 

• ik = 3 ii the kth co-corner contains a non-doubled x , 

• ?fc = 4 if the kth co-corner contains a doubled x and the kth corner is empty, 

• ifc = 5 if the kth corner contains a non-doubled x , 

• ifc = 6 if the kth corner contains a doubled x and the kth co-corner is empty. 

For example, the two surjective pretableaux in Figure 5 have respective profile 
(5,6,3,1) and (3,3,6,4). 

Lemma 5.6. We define the matrices Ni = A(B + xI){B + xl), N2 = yz{A + /), 
iVg = z{B + xl), N4 = zA{B + xl), N5 = y{B + xl), Nq = yA{B + xl). For any 
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{ii, . . . ,in) G {1, . . . , 6}", we have 

(28) ^^mi(T)^fd(T)^snd(T)-mp(T)-fnd(T)-sd(T) ^ ^^|^^^ . . N,JV) , 

T 

where the sum is over T E Sn of profile (ii, . . . , «„). 

Proof. We follow the same scheme as in Proposition 4.5, and use Proposition 5.3. 
Here, the surjective pretableaux of a given profile are in bijection with surjective 
tableaux of a particular shape. Rather than giving a formal detailed proof, we 
sketch how to understand the matrices A'^i through Nq, having in mind the proof 
of Proposition 5.3 and the way surjective tableaux are build recursively. 

• The kth co-corner and A;th corner correspond to the kth factor BBA in the 
word [BBA)"'. If these are empty {i.e. i^ = 1), we can remove the two cells and 
replace the factor BBA with ABB. With the terms xl and xl as seen before, 
we see that this case correspond to the matrix A'^i = A{B + xI){B + xl). 

• \iik = 2, i. e. the fcth co-corner and A;th corner both contain a x , we can remove 
the two columns containing these x , and remove the factor BB in BBA. But 
we need to distinguish two cases, depending on wether there is a third x in the 
same row or not, and if there is not, we also remove the row. This gives a factor 
A + I, and there is a weight yz because of the doubled x in the corner and 
co-corner. Hence this case correspond to the matrix N2 = yz{A + I). 

• li ik = 3, i.e. the kth co-corner contains a non-doubled x, we can remove its 
row and column, so the A;th factor BBA becomes a. B + xl. There is a weight 
z for the non-doubled x in the co-corner. Hence this case gives the matrix 
ATg = z{B + xI). 

• liik = 4, the difference with the previous case is that the x in the kth co-corner 
is doubled, so we do not remove its row. So there remains a factor A, and there 
is a weight z instead of z. Hence this case gives the matrix iV4 = zA{B + xl). 

• If Zfc = 5, this is similar to the case when i^ = 3. But the weight is y instead 
of z for the non-doubled x in the corner, and there remains a column of odd 
index so this gives a factor B + xl instead of i? + xl. Hence this case gives the 
matrix N^ = y{B + xl). 

• If ifc = 6, this is similar to the case when i^ = 4. But the weight is y instead of 
z for the doubled x in the corner, and there remains a column of odd index so 
this gives a factor B + xl instead of i? -|- xl. Hence this case gives the matrix 
Nq = yA{B + xI). 

When we form the product Ni-^ . . . Ni^ , it is clear that the matrix Ni^ will impose 
the conditions on the kth co-corner and corner, and hence (yV\Ni^ . . . Ni^\V) is the 
generating function for elements in Sn of profile (zi, . . . , z„). D 

Now, we can prove Proposition 5.4. 
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Proof. We have A^ = Y^^^i Ni, hence using Lemma 5.6: 

{W\N-\V)= Yl {W\N,,...N,JV) 

(n,...,i„)G{l,...,6}" 



E 



^mi(T)^fd(T)^snd(T)-mp(T)-fnd(T)-sd(T) ^ p^_^^_ 



TeSn 

This completes the proof. D 

From the definitions of B and A in (26), the matrix A^ defined in (27) can be 
calculated explicitly and we obtain the following statement. 

Proposition 5.7. The matrix N = (A^i,j)jjgN is tridiagonal, and such that for any 
i > we have Ni^i = hi and Ni^i+iNi^i^i = Aj+i. 

Proof. Straightforward calculations show that Nij = if |i — j| > 1, Ni^i = bi, and: 

(29) Nu+i = {i + l){z + y + i), Ni+i^i = {x + z + i){y + x + i). 

This gives indeed Ni^i+iNi+i^i = Aj+i. D 

As in the case of alternative tableaux in the previous section, the previous two 
propositions means that the continued fraction expansion given in (4) can be derived 
from the combinatorial interpretation in terms of escaliers, and the matrix Ansatz 
for escaliers given in Proposition 5.3. 

Remark 5.8. Observe that N ^ M since the non-diagonal coefficients are not 
the same, but the two matrices are equal after a permutation of the variables 
(x, y, z, X, y, z). However, there is a priori no simple way to link the matrices Mj's 
with the A^j's, so we tend to think that despite the similarities in the method, there 
are two really different ways to obtain the continued fraction from a combinatorial 
model using the matrix Ansatz approach. 

It is natural to ask if there is a bijection between 7^ and Sn preserving the six 
statistics for r„. From the fact that the recurrence relation is checked on both sets, 
in theory it might be possible to describe recursively such a bijection. It would 
be quite interesting to give a better answer to this question by providing a direct 
bijection between our alternative tableaux and Dumont's escaliers. 

Conclusion 

Our main result is the new combinatorial interpretation of r„ with the alterna- 
tive tableaux. We obtain two new proofs of Dumont's conjecture, with the same 
method applied to the two different combinatorial interpretations of r„. What it 
is interesting about these proofs is that they fit in the general framework settled 
in [1], linking J-fractions, operators satisfying certain commutation relations, and 
combinatorial objects. 
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